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ABSTRACT^ 

Summarized .are activities of the Wausau District 
Pubii'c Schools (Wisconsin) tovard developing a mpdel for cost 
efficiency analysis in special education. The model links 
input-output analysis and task ana-lysis features. Introductory 
information includes varying impressions of cost efficiency 
and the current status of project development. Discussed 
following elements of a cost efficiency study model: 
input-application-outputy feedback, outcome analysis, cost 
allocation, pupil accounting systems/ technical support systems, 
related technical issues. Application of these ele'ments in a cost 
efficiency system design for a local edijication agency is described. 
Reported are outcomes of pilot tests of the model and its components 
including simulation of resource allocation and design of a studelit" 
accounting registration procedure. Among project findings cited are 
problems encpunteyed such as lack of efficient pupil accounting 
systems-^-and ^reas in which progress w.as made — svch as development of 
task analysis classifications for special education. Appendixes 
iiiclude: documentation, inputs and outputs for simulated resource . 
allocations; a staff questionnaire; ajid a student registration form. 
(LS) 
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ABSTRACT 



This report summarizes activity of the Wausau District Public Schools 
(WDPS) to develop a -model for Cost-Efficiency analysis in special education. 
,y The resultant model links commonly used inputroutput analysis and task 
analysis features. This linkage is important if applications of cost- ■■> 
efficiency within- local education agencies- (LEA's) are to be productive. 

«.,'-> 

The central input-Output jnodel feature tha,t was adopted 'i=> a-modifica-_ 
tion of a four -elemen*t donceptual framework of education -subject to^conomxc 
analysis developed by R. A. Rossmiller and others at the Wisconsin Research 
and' Development Center for "Cognitive Learning. The four elements are: 
(1) resource inputs for the external-to-school environment, (2) components ^ 
•of the educational system, (3) outputs of the ecjucational system, and (4) 
'feedback. 1 'Linked task analysis elements involve (1) cost allocation, U) 
pupil accounting, and (3) technical support in knowledge &nd analysis 
capability forms. ' ' ' . . 

It is not possible for a LEA to independently perform advanced cost- 
efficiency activities in special-education. The knowledge base of the 
VvDical LEA is too restricted, most lack equipment or procedures for 
adv nc d dt-ta . analysis and too few students are present in ^Pe-^^^^^---- , 
finn nroerams for effective study 'of 'instructional program alternatives. . 
So^e^v r ' h^LEl's can engage in'necessary development of ^uvilj^r., ^ 
and cost allocation procedures if they have external help. Need^Q^ J 
supplemental knowledge .is continous. It is, perhaps best provided through 
noorrinafive effort at the state .level. "Needs for data analysis support 
' are intermittent. They can be provided by either a state university or ^ ^ 
•one of^^e limited number of educational support agencies that'has both 
a Lrge computer, and a well developed library for statistical analysis. 

^ ' This development effort has led to preparation of pupil accpuntdng 
procedurL for establishing a basic student file system ^his system wUl 
be^put into effect during the 1975-76 school year. 

initiated through --1 use of . . la.si xcatio^ ^-eru^:r ^ro-f es- _ - 

dgem nt^ ^^^^^^^ This program,;which enables exploration of 
the oroblem of 'program definition,- indicator- selection an'd prognosis for pro . 
.grL'impact eems I press the ability limits of special education personnel . 
to use ?heir judgements and values Co logically allocate. resources . 

-At its conclusion, this Title III development grant enabled the TOPS 
to organize its cost-;f f iciency development' within a long range framework and . 
to take necessary first steps for its implementation. Ad^it^ional f^f ^^^^ 
vork is needed with external agencies to supplement and hasten the progress that 
vould be made by the district alo.ne. 



Paper (Madison, Wis.: Wisconsin Kesearcn ana uev..j.upmuu. 
' Cognitive Learning,- University of Wisconsin-Madison, 1975). 
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INTRODUCTION 



Intent of Proposal 



The proposal for funding under. which this work was begun called for" 
the application of a system analysis format to model 4evelopment for 
special education program decision-making/ The proposal contained elements 
of uncertainty about the best ways to approach model building* , Among 
these uncertainties were' (1) the role multivariate analysis procedure^ ^ 
might? presently play, (2) the adequacy'of existing measurement procedures 
for student assessment, ^3) the nature of the different kinds of programs 
that might be proposed for development in special education, and (4) the 
relationships model development should have to still-developing mandates 

0 

of Chapter 89, Wisconsin Laws of 1973* 

Thus, as is customary in the application of development grants, it 
wak anticipated development of a cost-efficiency model would involve exam- 
ination of a large number of alternatives and''' selection of those which 
seemed most promising for subsequent useful applications • 



o 



Audience Perception of Cost-Efficiency Concerns 
The primary author .of this reporf found, a's work was begun on the ^ 

• a 
< 

project, that several individuals and groups of persons had very different 
impressions of what was to be sought through model development* Some per- 
sons seemed to believe the project would have iconcern for theoretical 

# 

model building along systems lines and that little if any practical test- 
ing of ideas would occur* A second group had greatest i-nterest in cost 
allocation. and seemed to feel this problem would constitute the primary 

— « 4 

* , ^ 

activity. A third group perceived the project to have greatest concern 

G ? 



for outcome analysis. Outcome analysis was perceived to be measurement 
0^ the performances of the special education students who were enrolled 

in programs, *^ 

A fourth cluster of impressions was that development of goals and 

objectives would be the primary concern of -the project grant. Program 

development would, it was surmised, dead to the initiation , of new (sub) 
programs within the e'icisting special,..education field, . ^ 

-5 In fact, the grant was not sufficient to accomplish all of these under- 
takings nor would its conduct- hsiv%.. It self, been -efficient if it sought to- 
do so. It is likely very substantial decisions about cost-efficiency can 
be made without the institution of fully detailed cost allocation proce- 
dures. There were no situations within the district where compayative 
outcome analysis for existing programs could have been justified. - Th.eore- 

" tica'l model building without concern for immediately usable outcomes would 
have boon unwise. Program development would have been excessively- costly 
to undertake within a framework for system development. The development ^ 
of goals and objectives is an important, distinct domain which cannot be 
entered without* awareness, of values individuals apply; development of 
goals and objectives is very time-consuming and a task which is never 
finished. 

The expectations of different audiences for accomplishments of ^this 
'project cbuld not all, be met. However, some 'progress was made<,in each area 
of initial concern and several topics not " specif ically addressed in the 
original proposal wer.e given detailed attention. Among these were student 
accounting, '"task analysis as it related to Chapter 89, and simulated multi- 
variate decision-making.^ These project outcomes were attained with aub- 
stantially lower expenditures .than originally budgeted. The Wausau District 
PubHc Schools liave reached a level of proficiency in cost-efficiency a- 
nalysis well advanced of the position they occupied one y.ear earlier. 

7 3 „ 



' Status of Project Development " •* 

A model has been developed .wlthii\ which the overall issues of system 
analysis for special education in tha'WDPS can be oriented. Specific steps 
'have been taken to facilitate long range development of additional capability 
for cost-efficiency analysis. New student accounting procedures are being 
adopted. They Crill enable more efficient record keeping in specia-1 education 
and will "facilitate ipngitudinal 'follow-up of students as is uSually. necessary 
'for effective output aiialysis to p.ccur.^ 

The district has accumulated- useful expedience by staff members in ^' 
analysis of technical problems associateci with cost-efficiency. The staff 
has had 'initial 'opportunities to reflect upon problem^ of program definition 
for cost-efficiency purposes and of output indicator determination. This 
increased staff experience is a necessary precedent to continued work in 

outcome analysis. 

In^roved understanding of personnel -time allocation during initial 
phases of conduct of Chapter ,8,9 activities has been accumulated. Because 
Chapter 89 implementation calls for important changes in proportions of . 
personnel time allocation to tasks over the first sever-^l ,prograra years 
^this information alone, updated and refined, is^ sufficient tp engage in 
many ei^f ectiyeness analyses. It is anticipated fh±s type of subsystem 
study will receive closest attention during the. next fiscal year. 

The \ms have also buiit^increased'^sensitivity to other areas where 
xost-efFiciency concerns may be directed without 'adoption of a full scale 
cost allocation. procedure. Transportation is* perhaps the most important 
of these other areas. " • • ' . 

St 

External resources are needed if there is to be rapid progress in . 
cooprehensdve full scale cost allocation and. If indicator selection for 

• ' 8 A 



pupil outputs is to proceed rapK^Ly. External support is also needed if • 

posslbi; benefits from simulated cost allocation through 'prof essional judge- 

ment are .to be exhaustively studied. This planning project concludes with J 

■ pi^ogress having been made, and progress continuing/to- be.ntade. Understandings 

Were acquired which will permit application of additional 'resources lo be 
/ . . ■ ' 

/made directly to development areas judged .by this study .to -be potentially, 

t. ' • 

^most profitable. 



El52MENTs' OF A^COST-EFFJCIENCY MODEL 



The term "model" as used in this^repbrt refers to coAsideratlons ■• . 
•and operations involved in special education cost-ef f fciency work, ^^hen 
used this way the term does 'not imply the existence of rigoirous, empirically 
based 'relationships as does model bui-lding in the physical sciences. „_ 
Instead, figures and narrative are used to produce a reference containing 
Ideas and .guidance for the development of capabilities to conduct cost- ^ 
efficiency studVes. Two processes (the hap^ings of 'l.roviding special 
education) and products (the outcomes of those happenings) may have costs ^ 
efficiency studies. Processes (the happenings of providing special 
efficiency analysis may . and usually will. 4iave concern for only subsets 
of processes and products. In the sections that. follow features of 
model-building relevent to special education are described and related 

V 

technical issues are emphasized ♦ " ' 

* % 

' Inputs-Application-Output • - — ^ 

One of the best kno.^ early system analysis models for education was 
. the student^-chal^ge model of .an educational system presented by Henry S. 
Dyer- at the 54th Annual Convention of the National Association of 
Secondary School Principals in February 9. 1970. Known to some as 
•'Dyer's wheel" it focused on student .characteristics 'before an^ after 
. conduct. of an educational process or 'program. These "before" and "after" 
characteristics constituted, repectively. the i-nputs and outputs of his , 
thinking at that time. The^ educational process- was perceived to .be 
tomersed in br rimmed by influences cau'sed by conditions in the home, 
. the community and the schoo'l setting ^itself Tb.e model £hus attented to 



\ 
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InaMUty to understand educational ou.co,.e_s without giving consideration 
to conditlc'ns external to an. i».diat,e act of instruction. It depicted 
interrelationships beiween the educational process and the external con- 
ditions , a £ea;ure that -u'st of necessity be included in any »odel devel- 
opment for cost-efficiency «udy, in special education. A copy ot Byer's 
'Wheel is - includcid as Appendix A. 



Feedback 



' . A u.r nvpr was intended to emohasize. how school 
The TOodel presented bv Over was xnut-uu^u 

^ -r^finpnces onlstudent performance \within a total 
processes produce influences ou^ ^ , r ^ 

fa.ily and co™,unity setting. Ihe author. vho has' perhaps .ost greatly 

influenced thi;king about, how information about school processes and 

.heir outcomes .is used to mate decisions is, Stuf flebeam. Bis ideas have . 

been disseminatid across this nation, usually as .the CIPP (context. 

. .^ ' The CIPP Model,. which was developed 

input, pVocess. product) Mod?l. "e -ii- , 

. in the mid-WSb's. was usually presented as a series of interconnected 
circles- with the circle used to illustrate how information-acuired through 
.valuation would be cycled back to -different users. , Among the'recognlzed 
user groups'were project directors, local education agency administrators. 

' , agency administrators and others. Concern for feedback . 

State education- ageui-y 

is a desireable flature to 's.ek in model develptmen-. for cost-efficiency 
study m the field of special education. Some of the variety of thought 
about feedback and decisUon-maKing that .xis.s can be found in Chapter 3 ^ 



;^;^;gXwrshington. D.C.. February 197 
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of a bpok edited by Stufflebeara and^'others ^'^ ' * . . 

Outcome Analysis 

* 

For effectiveness to be studied in a fullj^ meaningful manner' it is 

necessary that outcomes* be measured. It is also important, if programs 

are to improve through change,. that outcomes of -different ' types of programs 

\ ' ' , 

be detyirmined with sufficient precision that differences in those outcomes 

may be reliably specified. Be'cause of the small numbers of students 

enro?.led*in special education programs this is a very serious problem. 

As is common i^ other fields of education, special education lacks .^^^ ' 

* • 

precision in its statements of goal and objectives. Furthermore,^ 
measurement procedures that are adequate for determination of differences^ 
.in .program outcomes in other fields may be unsuitable in special education 
because of the slower rate of progress achieved by special education 
students, Feldt has presented an interesting article that deals with ^ 
this practical problem. Through study of pupil and cla^s norm data 
acquired by administration of the lowa^Test of Basic Skills (196A) he^ 
drew the following conclusion about minimum sizes needed for treatment 

groups: ' . ^ 

With ''.highly effective" treatments and sim'ple random assignment 
. '-of subjects to conditions, 60 to 85 subj.ects is derived as the 
* m^.nlmum number per group. With "m9derately effective" treatirfents,** ^ 
the .minimum number is 235 or more.,. Use oFstratified samples^ , 
.reduces the minimum by 15 to AO percent, )J * * 



^ Stufflebeam, D.L. , and other's. Educational Evaluat ion for Decision 
Making, Itasca, Illinois: F.E. Peacock, Inc., 1971, pp. 49-105* 

* Feldt^ L.S., "Wiat Sise Samples for Methods/Materials Experiments?" 
Journal, of: Educational Measurement , Vol. 10, No-. 3, Fall 1973, 
pp. 221-226. ^ ^ . ' • 



Cost Allocation 

- Cost alloca-tldn Involves the Identification of resources committed 
to"a ptogram. the determination of their monetary value and placement 
or crediting of the costs with program features. In educational programs 
by far the highest -fraction of costs are for instructional personnel. 

Financial accounting handbooks arc prepared by' state departments of 
education to describe and control classification o.f expenditures of 
greatiest concern. An important feature of these procedures is that costs ^ 
are «not directly related to programs such as those operating^ wiphin 
speciai education; ?.t ,has be^' very linusual.f or costs to be- r^ated 

directly 'to iri,aividu'al pupils. \ • 

. Cost-efficiency* canno£ be fully studied without provision for more 
'refined -allocation of costs to pupil's learning experiences than has been 
.the case to date- -Therefore, ^ost-ef f iciency model development cannot be^ 
,,-slxct^essfuily undertaken without special attention being gxven to this 

problem.^ \^ ^ ' ' ^ 

' / "Pupl?. Accounting Systems ^ 
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Many of the..goals of "education call for -the production of lasting 
effects on life styles,' values and accomplishmehtfid^f students. Outcomes 
of educational processes must. have durable effects upon student performance. 
Therefore, to hive successful arialysis of the outputs of an educational ' 
'system It is essential that the evaluating agency be able to monitof student 
'performance over time. "For- this reason, because educational outputs 
cannot be analyzed without fconcern. for interrelationships among home, 
- community ^nd external school variables which impinge upon children, an^ ^ - 
because cost allocation' should possible be made to individual students - 
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.8 units, model development for cost-efficiency must also provide for 
development of efficient student accounting procedures. 

« The rapid grovth of computer technology and accumulation .of experience 
vith pupil data systems in the largest local education agencies of- the , 
nation make possible the adoption by agencies without pypil accounting 
systems of the best of procedures developed at othe^ local and state 
agency si-tes. No fully comprehensive ^student accounting system exists' ^ 
but this critical cost-efficiency system component does not have to be, 
develop.ed^ in entire^y; " elements can be selected and adopted from among 
those used -by other agencies. ^ ' , ^ 



Technical Support Systems 



- -The development of a cost-efficiency model for a'local education 
agency is, as the title ,bf the activity implie^, Essentially a "basic ^ 

■ research and development endeavor. 'Xs. is the ca^e^ith other research 
'and development efforts the local edu^cation agency cannot be expected, 

by itself, to^ generate all of the^ necessary parts of^ the system- Two 
particularly critical support: elements are needed: (1) knowledge, and 
(2) technical service support/ The LIA cannot,, for example, generate all 
basic information, it needs about 'program impacts. It canhot s^lf- 

■ sustainingly engage in acro.ss-the-bpard, basic research stndies of program 
' Impact. It cannot' by Itself generate teacher effectiveness studies- nor 

can it evaluate all materials and related procedures available to it f or , 
use. Universities and other agencies produce the preponderance of new 
information used by LEA's. Therefore, the LEA' has continuing dependance 
on external agencies for efficient transmission of new -knowledge to it. 
• 'in addition, the LEA must have technical service support. It is not 
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profitable for the LEA to maintain programs or facilities for complex data 
analysis-t/or does it possess capabilities to evaluate prospective benefits 
of" new technologies. Thus the LEA must join with others or receive techni- 
cal support from universities and other external, groups. Development of 
a comprehensive cos^-ef f iciency model for special education should take 
this need into account. 



Related Technical Issues 



• In the. preceding five' sections of this chapter the most important 
topical concerns for model development were briefly described. Several- 
additional issues of an essentially technical nature must be kept in mind 

'J * 

as model development proceeds. First, is the current status of pro- 
gram planning budgeting systems (PPBS) development." This movement, which 
grey rapidly in the late 1960''s and culminated in an impressive effort 
to institute such systems in the -State of California seems now to be in a 
(temporary?) .state^ of contraction. The development of PPBS Constituted a 

r n 

formidable technical problem. Before its technical features could be 
resolved to the satisfaction of all interested, parties the California 
Legislature abandonc|d the program. Conflicts of value among political 
coalit-idns may have been the primary contributors to this outcome.- However, 
other groups once interested in the promises made for PPBS^ have also been 
unable to initiate and maintain rapid progress in this development area. 
Readers interested in better understanding how value conflicts may • 
frustrate technical developments and applications of systems for optimum 
resource allocation may find an article by Klrst interesting to read. 5" 

c 

^ Klrst, M. W., "The Rise and Fall of PPBS in .California." Phi Delta Kappa, 
April 1975, pp. 535-583. 



■ A second important technical issue has concern for the Importance ^ 
of being able to do -follow-up studies of student .perfonnance. Most of 
the educational research that has been conducted to date has been cross- 
sectional. .Gross-sectional data is collected at a point in time; for 
example, a measure of the performance of all- hearing-impaired children^ 
in an instructional unit on June 5. 1975. In 1970 Hilton and Patrick 
published an important and definitive article on this topic. They compared 
.three sources of data for studies of growth — matched-logitudinal, 
-^nmatched-logitudinal, and cross-sectional. Their study supported the 
conclusion that matched- logitudinal'-data, with matching performed to link" 
the different scores of individual children over time with each other, is 
more reliable <and therefore more likely to produce valid analytic results. 

A prior, related study vas conducted by Dyer, Linn and Patton.. Their 
concern wasleSs with ijidividual performance and more with the prediction 
of school system means on achievement tests. However, the results of a 
comparison of four methods of obtaining discrepancy measures led to a. 
•similar conclusion'. Their study involved data collected over, a three ^ 
year period of schooling. 'Theit conclusion was: ".. .discrepancy measures 
based on ui^matched-longitudinal or cross-sectional samples of students 
cannot* be regarded as reasonable substitutes for discrepancy measures 
based on a carefully matched-longitudinal' sample. 7" These two articles. 



^ Hilton, T.L., aTd^ Patrick, C, "Cross-Sectional Versus Longitudinal 
Data: An Em^Lrical Comparison of "Mean Differences in Academic 
Growth." Journal of Educational Measurement, Vol. 5 No. 1, 
Spring 1970, pp.^ 15-24. 

7 Dyer, H. S. , Linn, 'r. L. . and Patton, M.J. '''A comparison of «f 
' of', obtaining Discrepancy Measures Based on Observed and Predicted 
School System Means on Achievement Tests.'' ^ipil^al^^^^on^ 
Research Journal, Vol. 6, No. 4, November 1969, pp. 591-605. 
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which were prepared fro^m work with real data, strongly suggest oany ^:ost- ^ 
efficiency procedures will^ not themselves be efficient or productive if 

they do not. provide for the collection of longitudinal data. 

At' the same'time the previous articles were being prepared Werts and 
Utin.^ working on the s^e proble:=s. prepared a general linear model for 
the^ study of growth. Their paper had concern for the identification of 
cauLl relationships in data. The paper^concludes with a reminder thaf 
to be able to perform statistical, computations o'n data may' provide us with 
ver'y little useful understanding. Artifacts of both measurement and the 
analysis can make a model treacherous to- use. 

Comprehensive cost-effectiveness analysis of alternatives for special 
education program implementation will almost always involve quasi- 
expemmentation. This .endeavor involves, along with 'the technical issues 
already cited, need for great sensitivity on the part of the analyst 
for th^types of outcomes that may o'r may; npt.be,plau6ible when new 
programs are initiated. 'A particularly rich referent for these under- 
standings is an articlfe by Wiley and Bock. Ihese writers were interested 
In the sizes bf groups needed to draw conclusions about innovative, pro- 
.grams but also" wished to understand what ' contributions instruction might 
make to. improve student performance in such areas as'spelling. arithmetic 
. computation, social studies and science. They concluded that a study 
involving as few as. six schools with two classrooms in each school could 
Wule detection of a mean differ en'ce between experimental groups of as. 
much as :3'2 grade equivalents. Whereas a building might be effective 
In the product-ion of computational skills and a school district 



8 Werts'. C. E. and Linn. R. L.^ "A General Linear Model for Studying ^ 
Growth." P. ycholor.ical Bulletin . Vol. 73. No. J.. 1970. pp., 17-22.. 
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Bight have unifonnity within it in computationa-1 attainments, performances 
in areas such as paragraph meaning might be much more variable among 
schools with comparable efforts. The influences other than 
the classroom .performance of a teacher can contribute heavily to stu.dent 
' performance. Summary reflections by the authors can provide stimulating 
reading and useful insights concerning the prognosis for cost-efficiency 
study of program outcomes when a type of program has been carefully 
formulated and described,^ 

Perhaps^the outstanding authority in the nation with practical 
experience in the study of change is Donald T, Campbell who haS 
been interested in quasi-experimental models of the ^type that mos.t often ' 
must be used in special education cost-ef f icdency work and has written 
extensively on the topic. ^Most of the analytic concerns that haVe been 
identified to,.date were summarized by him in an address to the 1970 
Invitational Conference .on TeWing Problems; the Contents of his address 
contain many valuable ideas. . . . ^ ' ^ 

- One additional 'article may be of interest to persons contemplating - 
the values of benefit analysis. This is a 'reflective discussion by, 
" Lohnes that arose. from general concern for the schooling of intelligence. 

If cost-efficiency procedures are to be developeS for-, special education, 
■ reflections such as his should be heeded.' They suggest cost-efficiency 
analysis of outcomes frequently" may not be profitabl-e. He noted:. 

\ Instead of s§?eking a best instrOc'tional system, research might 

^ , better seek to reveal the correlations between degrees of 



9wiley David E. and Bock. R. Darrell. "Quasi-experimentation in 

Educational Settings: Comment." The School .Review , Winter 1974. 

lOcampbell D. T. "Temporal Changes in Trealment-ef f ect Correlations: 
/'a Quasi-Experimental Model for Institutional Records and l^ongitu- 
^ dinal Studies." Proceedings of the 1970 Invitational Conference 
on Testing, Problems , Educational Testing Service, Princeton, N.J.^ 

i97r, ^ ■ . . 

• J8 , 
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implementation of various treatment dimensions and degrees 
of achievements of Various . types . It m.ight also try to dis- 
cover whether these treatment-outcome correlations are in- . 
fluenced by non-linear involvements of organismic inputs makxng 
moderator effects or treatment-aptitude >teractions available. 
This implies analysis of canonical correlations, multiple partial 
-correlations, and homogeneity of regressions ^y^^^^""^' J^"^ 
first it implies that dimensions of treatment programs be 
conceptualized, scaled and measuired in school trials. 



llLoHnes, Paul R. "Evaluating the Schooling of Intelligence." Eddcational- 
Researcher, February, 1973, pp. 6-11. • -• 
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A COST-EEEICIENCY SYSTEM. DESIGN- FOR A LOCAL gDUCAT-ION AGENCY . 

^ Overall Design ^ , 

The overall cost-efficiency study model that was developed' in chis' 
. .project involves a synthesis of selected t^sk analysis and input-output 
elements. The 'f oca.1 element in the/model is a modification of the work by 
Rossmiller, Marinelli and Geske . . n^j^^^ activities of a cost-efficiency 
analyst must show concern for these program,. constituents. 

For the' efficiency analyst to study. inp'Gts' to special .education 
programs, describe the application -of resources and produce information 
to bevretumpd to others -about the merits of program outcom|s, it is 
necessary that support system^ be available to the analyst.! Accounting 
y ..procedures used in the agency that emj^loys him must the^sel^es permit 

efficl-ent^allocations of cost^ for the time, personnel and other resources 
that are^pplieU- in programs/" Cost allocation procedures should 'be fully 
compatible with those^o"f>the_State Education Agency (SEA) . Those 
• currently in operation, in ,kscons^~Vfe-not. fully adequate for needs 

of the analyse nor are those in most other states. Theref ore,_imp6rtant 
cost allocation development needs exist in this and similar LEA's. 

Pupil accounting system support is also needed by the analyst. In 
a very small LEA it is/.relatiyely easy to maintain and access records 
for the small numbers/of special education students who are pre'sent. 
Their p-erformances oyer time, in response to the applications of^virlous 
resources on their behalf, can be efficiently recorded and stored al- 
though, even in the smallest agencies, lack of a comprehensive student 
accounting system may make the preparation of summaries of data for-'- 
. "l2Rossmiller, R.A. , 'Marinelli, J.J. and Geske, T.G., op cite. 
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.nalysis'or related reporting purp.a..esj^:dif f ic^ accomplish. 

In medium and large size LEA's the development of basic district- 
vide pupil accounting. procedures and supplemental but linked procedures 
,£or special education will usually'be a profitable undertaking. These 
systems should^ include provisions for retention of family data associated 
with students, general demographic information pertinent to them. 
^ educational program information and. info nnat ion necessary to engage in 
'efficient follow-up study of their progress. Student anonymity must, also 
be protected, 

A final necessary ingredient for the .cost-efficiency analyst to 
.engage in his work" is the 'availability of technical support from 
external.. agencies. This technical support will provide him with 
knowikge necessary to the conduct of his work and with support in the 
"actual conduct of analysis Sf data. Assistance in report writing may 
"^also be a necessity." Technical sppport may be provided by a single agency 
(The Institute for Educational Research at Downers Grove". Illinois, is 
illustrative>_or by a variety of agencies such as universities, state 
. departments of education or independent, nonpublic bodies. 

Figure 1 simply summarizes .the essential work elements involved in 
application of this manner of thinking. The model that .is presented 
, depicts,.primary concerns of the analyst with input, application, output 
and feedback. It represents his dependence upon related operations by 
arrows pointing out the flow of information to him from cost allocation, 
''pupil accounting and technical support activities. It also indicates 
that the ^Information generated as a result of the activities should flow 
to excema^encies. In particular, knowledge should be transmitted 
out to the tech^^i^support agencies as part ,of.vhat should be formal, 
collaborative relat^ips among them.- the.-Jnalyst is unlikely to have 
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primary administrative responsibility for the accumulation of cost 
allocation information, could not hav/administrative control over 
external technical support, but might retain authority over pupil . • 
accounting operations. In the sections that follow the primary and 
secondary elements of this system. are given additional elaboration; 

"^^^ Input-Application-Output-Feedback ^ • ' , 

The core element for system analysis must, as noted earlier, 
provide for analysis of inpSt and outinit of an 6petating,_^educationaV , 
system and for the recycling of informa-t^on that is acquired. Figure 2 
provides a more detailed visual record of the four components denoted in 
the focal unit of Figure 1.' It records, in parallel lines for rectangular 
blocks, the most important features associate with inputs, resource 
application and output^?. These three component features are interlinked 
through the feedback activities of measurement, analysi? and reporting. ^ 
' ■ , ilome and community ,; the lattfer through each of its immediate local, 
- regional and state characteristics;, -constitute the external environment. 
. From/ this 'env/r;&nment are contributed^ knowledge, societal values And 
goals.. The population of the community served by th^ 'educational system 
•posse'sses characteristics^ of size, -density , social organization of . . 
families, mobility, etcetera f these characteristics can be described as 
demographic and social-economic. 

The' external-to-school environment also possesses important character 
istics of economic output and income. Some school buildings and" entire 
educational agencies have. very low economic outputs and incomes; for 
others, especially those commonly referred to as "affluent", economic ^ 
output and/or family income can be very high. These characteristics -are., 
usually associated with very different student ap.titude^ for academic 
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performance' Their importance caij be discerned by reference to the large ^ 
numbers, of educational programs generated far the Gconomically needy within 
the past*decade. The importance of the external environment is perhaps 
most often discussed ^in relationship to economic features with regard to , 
influences on both 'total education system perfpcmance and the performance 
of 'Students within, the System. 

In its largest component feature, that for th.e educational system as 
a whole. Figure 2 contains what may appear to be an inconsistency., The 

term "resource inputs at school"- is used, This'distinction is a common 

• ' • y * 

one within educational, literature. The external environment provides re~ 
/ . • 

sQurces necessary to acquire such material resources as learning aids and 
educational facilities. It also provides resources necessary for the em- 
ployment of staff and school management. It sets aims, priorities and con- 
trols through the functioning of Boards of Education, Once aims, priorl|- 
ties- and controls have been applied and material resources have been as- 
sembled, there exists a school system. It is this body, "our school(s)", 
which in turn initiates programs for students. Thus it is possible to think 
of such an assemblage as itself a resource input to instructional .processes; 
the student is in turn the recipient of these applications^;----'^ 

Efforts to improve the performance ability of teacheri^ through inser- 

, ) 

vice programs is illustratfve of jthe application of externa^ resources to 
assetnble an internal resource of a higher quality with the expectation 
• teachers will, in turn, apply their improved skills to the improvement of 
instruction. The apparent inconsistency cannot be wholly eliminated by 
reflection on this example but no more useful classification of system 
components and functions seems yet to have bee devised, 
' ' The application of resources by an Educational agency is most often 
thought of in terms of the instructional process, Schpols provide pro- 
grams for children which involve instructional units of time, instruction- 
al materials, grouping patterns and space use. Students are assembled; 
1 ■ 2<1 20 ■ 
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Infoi^pation about thc^ir performances are used to modify instructional 

programs for them by "indiYidualization of in'sfcruction; 'staffing patterns o 

Ti^'S^i^tioi^s^-"^^^ provide -goc^d instruction/are brought 

toge^er; administrative patterns are desi|ned*-to- pr:ovi^e^~neq^sary 

support services. Additional supportive services are provided in T — 

such areas as special education, guidance and counseling.' Monetar^ and. 

nonmonetary costs are involved. Reso.iirce application by the educational. 

system is, in common terminplogy, "what the school is' doing." 

, A rapidly gro\>?ing bo^y of literature analytically describes the 

products of resource application. Cost-efficiency analysis has greatest 

concern for what- indicators of outputs of the educational system can 

contribute to understanding of 'what education is accomplishing. Figure 

2 depicts these indicators as being, classif iable' into shdrt range 

and long range outputs. In the- simplest sense, short Tahge outputs ^ 

are those produced' during the interval the course or program is 

offered to its students; long range outputs are those which remain or 

■ \ • . ' • ' - . . , 

accrue over additional dayg, weeks or years. Tl^e most broadly 

descriptive terras used- for the class^Lflcation of human preformances 
'are the terms cognitive, affective and psychomotor., While thiree other 
, . terms, understandings, attitudes, and skills, have ^1 ess precise meanings 
they describe the same general perf ormance'^areas , ^ * , 

The values society holds for its >iembers contribute innumerable 
ideas about what the long range outputs of the school system should be- 
Jujhe great diversity they possess has necessitated use of broad goal ^ , 
"^'^ statements for school output such, as ''preparation for' life", ".a 
productive member of society", and ''a productive individual". No 
classifications with the clarity of the three available for short 
range outputs had been prepared for long range outputs. Of course. 
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the taxonomies for short range outcomes are/ applicable to long range 

outcomes but because long range outcomes and goals are much moxe 

Ukely to be stated in general terras^ indicative of performances at the 

higher levels of the taxonomies, r^fiiviment needed for the latter to 

be 'Classif ied doe? not exist. . As a result,. Figure 2 uses a single 

stem, "improved capacity to function in society 'as:" to stat/e five 

1 ' * * - ' ' ' . 

-general' outcome gbais. It is necessary that more attention be - 

given to imprpvertient of long\range goal specif ications.... Without 

improvement the assessment ot long range outcomes will usually be . 

, paorly performed. . , * - ^ . ' - 

, * The 'fourth component of^Figure| 2; that for feedback, is briefly 
described through the* u&e of the. three words* assoc^iated with it 

"([measurement, analysis and reporting).. ' The« figure is inter*Hed to 
show that feedback information is acquired af the^time of resource 
appiicatidn anfl 'even before. Feedback on outcome^of educational 
performance and information about, process is used to make^ modif icatxons ^ 

'in resource, input allocation from- 'the external environment, from" the • 

• resource pool accumulated at 'the school,' and to make changes in the 
application of resources. , ' . 

Because figure 2 does not by itself provide sufficient elaboration 

*on -activities associated with the preparation of information through 
to uae of . feedba;ck. Figure 3 has been prepared to meet that need. . , 
The key feature of .this figure is that it responds to the fact decisions 
tend to be-'made by sele'ction from among alternative courses of action. 

two programs (or more) have actually been conducted ,f£he evaluation 
of their outcomes begins with identification of the external resources 
made available for their conduct. ' Information about external resources, ] 
school resource inputs and the actual conduct or processes of th.e programs 

' 27 ■ 
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are accumulated for subsequent analysis and reporting. Outcomes of 
each program option tend to be analyzea -separately . ■' In some instances 
results of analysis for a particular program are of interest by 
themselves but in most cases information about alternatives is ' 
compared 'by statistica-l or judgemental processes. The information 
that 'is generated -'is subsequently assembled^ in a formal report and 
distributed back to the persons who will use it to make decisions.. 

Jor the evaluative and feedback activities indicated in Figures 
2 and 3 to be" conducted it is essential that the cost allocatign, pupil 
accounting and technical support systems function. Cost allocation 
activities provide financial information linked to and descriptive * 
of inputs. -Pupil accounting activities contribute information about 
resource inputs and resource 'applications. Pu^il accounting also . 
contributes capabilities for linking input and process information 
with output information. Information generated in pupil accounting 
and in- cost allocation will frequently go;directly into the feedback 
mechanism without being linked to output information. This type 
of information transfer is essential if processes are to be studied 
and modified before outputs are subject to study. 

Technical support is, as has been noted earlier, essential for 
conduct of each of the other classes of activities. It provides 
information for the classification or reporting of inputs, design 
and implementation of applications, and for the organization of 
^comparative analyses. Technical support informati6n will appear 
in feedback reporting; . without, it the conduct of analysis will 
sometimes be impossible. The arrows in Figure 4 are meant to depict 
how information-producing activities are linked within the total 
model generated by. this project. 
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Cost Allocation System 

^* 

■ Three procedures (models for cost -allocation) have been given 
greatest consideration in this development effort. The first, and , 
the only one in use, is that defined by the Uniform Financial Accountinp, 
Handbook for Wisconsin School Districts , 1974 edition. This document ^ 
provides account classifications for receipts and dispersements 
definJLtiohs *for the classifications of expenditures and for their 
analysis in Wisconsin, school districts . ^^^ 

A. second procedural guide that has beeii considered is a set of. 
revisions for the first reference document currently in preparation by 
a committee of members from within and without the Wisconsin Department 
of Public Instruction. Revisions being considered for the existing 

state code vjould enable much more direct allocation of costs to 
programs. Deficiencies in existing procedures, which hive greatest^ 
practical significance will,-, in a large p.art, be , corrected- when the 
new procedure's are adopted. 

A third reference that has been .studied is a report prepared by 
the accounting firm of Ernst & Ernst. Erhst & Ernst was retained 
by the Governor's Office of Human Resources, State of Illinois, and 
also partially funded by the United States Department of Health, 
Education & Welfare, Bureau of Education for the Handicapped, to 
undertake study of the cost allocation problem as it exists for special 
and regular education in school districts representative of Illinois^^ 
and the nation. The accounting understandings they brought to their 
task were strong and led to the development of procedures intended for 
practical application in cost-effectiveness work. 

l^isconsin Department of Public Instruction. "Uniform ^?;"^"5^f 

.Accounting Handbook for Wisconsin School Districts." 1974 Edition. 

■ 31 27 . 



The fcmst &■ Ernst model for determination .of costs of special , 
education results in the production of reports- on the amount of education 
delivered -through the use of "^ime and money reBourges. It^does not , . 
report on the effectiveness of the services that afe delivered:" There- 
fore,, as, is the case with the other two procedures for cost allocation, 
it can serve as' only | component' of an overall cost-efficiency model. . 
- The Ernst & Ernst model is based , upon the use of a defined ■"student 
educational unit" (SEU). The SEU is defined as a period of ten minutes, 
during which time a .student is served by an educational agency. Possible 
use Of this unit on at least M occassional ^tasis for the determination 
of cost-efficiency of LEA program^' is appealing. Clear examples of 
What might be used were given in the Ernst Ernst report. However, it 
serves best at this' time, to be referenced as a standard for long term 
cost a'ccounting development rather than as an immediately useful inodei.l^ 

In summary, we propose that a cost allocation system of the -second 
type, that 'currently in development for application by the DPI, is _ 
a preferable model to adopt for the next several years or more; in 'the 
, long range such' a model should be supplemented with selected features of 
the Ernst & Ernst model,. 

Pupil Accounting System 

The National Center for Educational Statistics of the U.S. Department 
Of Health, Education and Welfare/Education Division (DHEW) in 1974 
revised its student/pupil accounting guide . < This document provides 
standard termonology and guidelines for managing student data in 



l^Ernst & Ernst, "Report a Model for the ~ Determination of the Costs of 
Special Education as Compared With That for General -Education 
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elementary and secondary scliools and other educational agencies. 
However, while it can serve as .a model for the adoption of termonology 
and general management of information it does not provide a model of the 
actual- form design that would be appropriate for a local education 

agency. ~ ' * ' , 

< . Fortunately, the nations largest cities and various educational support 
agencies^ for sntaller school districts have 'developed functional pupil 
data systems. The Philadelphia schools and others considered the 
6ysteii\s to be indispensible/ For example, the Di.rector, Division of 
Administrative and Survey Research Services, the School District of " 
Philadelphia, included the following statement in a presentation he 
made to the Americal Educational Research Association in- April, 1975: 

.;--"As the size of school 'district increases, so do the number 
and complexities of the problems encountered and the need 
to have a functional automated pupil data base becomes a 
requirement of sound educational management. Such a 
data base system can no longer be considered a luxury. ^ 

Tlie more ambitious an LEA is with regard to cost-efficiency studies 
the more valuable an efficient student accounting system becomes. It 
will be of great value to any agency enrolling ten thousand' or more 
students and may be quite valuable to agencies half its size. 
The Philadelphia pupil accounting system employs a pupil identification 
•number to coordinate the storage of informatlpn about birthdate, sex, 
ethnic group, housing location, and other elemental types of infor- 
mation. • A more comprehensive system for accumulation of basic pupil 
file information is used by most other school districts. The Minneapolis 
schools, 'after careful study of systems used-by the Seattle, Milwaukee 

^^Wisconsin Department of Public Instruction. "Uniform financial '.:, v> 
Accounting Handbook for Wisconsin School Districts." 1974 Edi.tion, 

^^Penry, Edward B. "An Indispensensable Tool for Research and Evaluation: 
A Functional Pupil Data System." April 1975, P- 2 ^ 
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and other school systems, have designed the procedures that were a^dopted 
as a aodel for this project.* In combination, the Minneapolis system _ 
and the U.S. DHW Student Accounting Manual have served as the pupil 
accounting system models for this project. . 

Technica'l Support System 

Inclusion .of a technical support element in the model that has 
been presented thus far is intended to provide a direct link of concerns 
for the performance- of knowledge and service-producing agencies in their 
intended support roles. Dissemination of knowledge produced elsewhere 
to LEA'S in a useable form is" also critical if cost-efficiency work is 
to be done at the LEA level. For example, the number of individuals who 
possessa particular handicapping condition within a given LEA vill 
usually not be sufficient to do com|>rehensive evaluative research with 
them. Indeed, the number available in even an entire state may be 
insufficient. Therefore, there must be ways by which state departments 
or other agencies can accumulate and disseminate knowledge helpful. 
to the LEA. Serious cost-efficiency analysis may not be possible for 
some handicapping conditions witl^out this type of coordinative involve- 
ment by an external agency; the LEA .jiust constantly be aware of its 

limitations in this regard. 

When research performed at other state and national locations 
has implications for an LEA's cost-efficiency development work it 
is important this information be efficiently disseminated to it. Thus 

> 

the LEA is dependent upon external agencies both for the production 
of knowledge and for the distribution of it back, to sources of data 

t 

♦Because a modification of the Minneapolis system has ^^^^^^ 

preformed and that modification is discussed later in t^^^^ "P°^^ 
' ".no further discription of the Minneapolis system is recorded herein. 



generation, -An LEA 'engaged in cost-efficiency activities must continuously ■ 
.retain strong ties with external sources of information. 

Most data collection and analysis services can be provided to an . 
LEA by a major university within its state. In some instances additional 
^ special service agencies (The Institute for Educational Research, Downers 
Grove, Illinois is illustrative) may provide these services. Instrument 
development, conceptualization of problems, organization of data for 
analysis, analysis itself and reporting are illustrative of this type 
of technical support need. Support must give capability for conduct of 
-the various types of multivarate analysis now available and, therefore, 
must be able to provide access' to, computing equipment with large , 
information storage capacity. 

The model that has been adopted by^this project provides no ' ■ 

additional figural summarization of elements of technical support but 
emphasizes that without this type of service cost7ef f iciency analysis 
is unlikely to be effectively carried out by an LEA. 

Internal Organization , . 

* J, ■ v{ 

- The 'organization of Figure" 1 suggests several alternatives for how 
the organization of cost-efficiency studies may exist within an ^ ^ . 
■educational agency. Cost allocation activities are ^eavlly dependent , ^ 

* • • J ♦ * 

upon the ability of the business services unit' to" provide cost 
informa'tlon; .therefore, thls'unit of the LEA must serve' a.- support. 

* function. . . ' ^ 

The delivery of external support services must be provided by the- 
state education, agency (SEA), a major university and/or perhaps a 
. ; special. external agency.* The best combination of external support ^ y 
services will .'tend ,to diicfer^for LEA's within different geographic areas- 
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Coordination of provision for these services in the most, efficient manner 
will often be difficult to accomplish. 

Student accounting* services may either be organized separately 
from the cost-effectivenesS' analysis operation or within the same 'unit. 
An organizational unit /Which has as its overall responsibility the 
accumulation of all information for professional and public needs may 
be most suitable for large educational agencies. Smaller agencies may 
find it best to develop an arrangement more unique to themselves which 
combined existing personnel, and organizational strengths. This' 
organization may or may not result in cost-efficiency work for special 
education being conducted by the special education department. 
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' OUTCOMES OF PILOT TESTS OF .MODEL AND COMPONENTS 

V 

The proposal for this project- called for extensive involvement of 
the special education staff of the district in efforts fco define programs 
and outcomes, the data from which could permit cost-effectiveness studies 
to be conducted/ Extensive staff involvement occurred with the involve- 
ment focused upon the elicitation of staff members' ideas most pertinent 
to model development and application. Staff involveihent focused on 
work" with the problem of program definition and on the problem of 
Indicator selection or development; it concluded with a survey of time 
commitments of staff members to classified task activities. 

As model development proceeded there was automatic involvement of 
external support elements, mgst iiiiportahtly the WDPI and the University 
of Wisconsin. An unexpected event was the initiation of development 
work in student accounting; the latter activity occurred as it became 
evident that existing information assembly procedures were wholly 
inadequate to the needs of a, cost-ef f iceincy analysis system. In the 
report sections that follow pilot activities are described and their . 
^implications are noted. 

" ^Simulation of- Resource Allocation 

During and jabout the year 1968, Yehia Badran, then' emiJloyed by 
Educational •]:esting Service, engaged in study of the resource allocation 
problem faced' by a large city with high ESEA Title I expenditures. He 
developed mathematical models suited to ,the problem and designed, linear 
analytic procedures for allocations of dollars to programs whey:i judge- 
ments of probable outcomes and their values could b'e specified by 
authorities or other interested persons. Badran' 3 work, which was 
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largely -theoretical, was adapted for computer application by Robert 
Patrick and Harry Hartnah of Educational Testing Service; their programming 
yas used internally with S^Ws of educators engaged in workshop study 
/of the decision-making' process. 
" Dr. John Cook of the DPF, upon hearing of the work of Badrah, 
Patrick and Harman, concluded the model and procedures. migh*t' have 
particular value for special education. He subsequently acquired 
a copyfof the program which he then modified for use at the computer 
facility of the University of Wisconsin - Madison.. A summary description 
of what he learned through 'its application was published as an article in 
"Bureau" Memorandum. 17 Cook's experience with the .procedures was not • 

■ I 3 

' exhaustive but it enabled him to .conclude they might be helpful to the 
decision-making process in situations where hard experimental information , 

. was not avialable but decisions had to be made by use of bast" combinations 
of judgement ajid information. 

The decision model described by .Cook provides a means whereby 
simulation of cost-efficiency work in "special education can be conducted. 
It is,,^in a sense, a technique for analyzing the function of an educational 
system. For it to be used, cost information must be available as actual ^ 
dollars or estimates; pupil characteristics must be accounted f|r; 
knowledge about the.' likely inpacts of programs must be referenced; input- 
appllcat ion-output system characteristics must be.'considered. Use of 
the- program culminates in the printing of recommendations for thg provision 
or nonprovislon of programs. These recommendations, which will usuaUy , ■ 
be seen to be 'based on fallible input, will tend to produce intense 
reflection followed by redesign of program and rknterpfetation of 
values or prpgr(jms. Jt provides "a means for engaging in cost-efficiency 

/ 

^^Cook, John J^, op cite* ^ . , 
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analysis at the highest le^l of available knowledge at a given time. This, 
in turn, enables identification of obstacles, to furtfier development of cost- 

effic^iency procedures. * ' . 

Five steps can be defined for preparation of * infomtation necessary^ 
for linear program analysis of progrart information by" the-decision model.. 
.The -first table reporta basic data descriptive of (1) programs among f ^ 
which resources might be allocated, (2) the maximum enrollment potentials 
of tho.se programs,' (3) the cost of providing a program, and (A) computation 
of the related cost per pupil. For this activity to be successfully 
carried out it" is necessary that the definitions of. programs competing 
,for resources contain sufficient cla-^ity to permit precise allocation of 
resources, to them. The definitions must make clear whether a teacher is 
or. is-not working on a program at any point .in time. It is also necessary 
at other resources called fdr in conduct of the program, such as 
uilding space -and- materials, be unequivocally expended for the 

program. Without this iri'Eormatioh .program costs cannot be determined , 
and, as will subsequently be noted, outcomes cannot be attributed to the^ 
program. An example for this and the other four tables was included ^ • 
in the article, by Cook which is contained in Appendix B. 

n 

Bcftween completion of this first table and later tables it is 
necessary to consider indicators of program success |<Jr",each program. 
This is the most difficult task involved. in the simula^ted .decision-making 
experience. What outcomes are -being sou&ht? \7hat measurement is ^ 
appropriate?. These questions and many more linked to them must-be 

discussed by persons using the procedure. The* issue" of norm^ref erenced^ 

' ■ - V ' 

versus criterion-referenced assessment may .arise. "The classification ^ 

' * : . ^y- 

'of -outcomes into cognitive, affective, and psychomotor will >Jpost 
surely enter into consideration. Finally, .a real or hypothetical , * . 
performance scale must be agreed .up one' as an indicator _,of 'performance. ^ 
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When indicators have been sel-ected and their meaning agreed upon ' 
by persons involved in the simulation experience it becomes possible , 
to proceed to additional input preparation. A second table is prepared 
to summarize the current status of progra^ if they already exist, or for 
their initial status at the beginning^of the progr^am if a new program . 
is contemplated. In this activity the proportions of students at 
different performance levels on <iach indicator;.and program^'are recorded. 
This constitutes inventorying ofTnitial st;udenfe^status. The proportionk 
■ must include all students who. might later be included in a functioning 

^ program^ 

A third table is prepared by consensus of the dccisior.-making 
group or by each individual member to\ summarize the values they ascribe 
to each of 'the. programs. This is prioritization across programs and 
is engaged in by assignment of a weighted- concern. to each program. 
Concerns are expressed as decimal fractions 'and" must sum to l.OO,.* ' . 

Preparation -of these- decimal'fractions consistitutes the assfembly of a 
/ ' relative rating of the importance of the different programs. ^ 

. . The fourth table that, is prepared calls for the ^judges to determine >. 

how important progress of a student from on^^ scale level to "another 

.' ■ " - . » ■ " 

may be. This is called prioritization within programs^ The prioritlzors 

.- .- consider" the meaning -of measurement at each scale level, its implication 

• i . ■ 

' for -the well-being of ,the 'student , the likelihood the student would 
-progress to the ne'xt level without an intervening program, and other " 
issues. At the coficlusion of their reflections they pripritize each _ 
scale level for each program by itself. -Decimal fractions are again, 
assigned with the total of the decimal fractions eqitelin^ 1..00 for 

*A difficult-to-understand feature of the statistical programming- is that 
one concern must usually be specified as .50. Tliis restricts the ' 
flexibility of judges in expression o& their values but does not 
seem to be a debilitating restriction for the program and procedure. 



each program. 

Completion of 'the fifth table will perhaps draw most decision-n^kers 
to the limits of their knowledge and their, capacity to make judgements 
about program outcomes. It involves two phases. In one phase thejudge 
.produces an educated estimate of the probability- students would move 
from one scale-level to another without intervention. For program " 
with which they are 'familiar they may reflect on each individual student , 
(if the numbers are small) and ask themselves whether the student .> 
would progress without the existence of th'e program. They will also ask 
.Vhether the student would, nv.r . defined interval of time, progress 
not at all or across one or even more levels. 

. ' In the second phase of this work the' judge will make the same , 
-decisions, for the students with the expectation that the program wolild 
occur; thaf is, intervention, as defined by the program would provide 
experiences for the student. The procedures are identical to those 
for completion of the .estimates without intervention. 

Coii5>letion of the fifth table constitutes completion of all informa- 
tion necessary for programming except one item. It ,is^ necessary . for - . 
the program to run successfully, that a statement of the amount 'of money 
available for expenditure in pro grams ''also be provided. This informtion 
. is usually readily available as a real of proposed bud'get allocation. . ^ 
• The data that have been prepared is keypunched for entry into the 
statistical run. The cost of a run invol^ving four programs and four 
levels per program is, at this time, about five dollars. The output 
Includes- a statement of the logical' outcomes of analysis, programs that 
should be implemented and the levels at which they should be implfeWted. 
A proposed enrollment for each program is summarized in the input 
information. Fin'ally, a cost for the total set of programs and for each 



individual program is also recorded. Tliis output constitutes the f eedr ^ ■ 
back information to >e used by whoanever is later engaged in the decision- , 

making process. • - ^ 

Interactions'^ with the special education'^staff of the Wausau District 

Public Schools were primarily concerned with decision,-malcing problems in 

situations where comprehensive input, application, output and analysis ^ 

studies were not possible* It was felt that the most promising area to 

try out a simulation would J^e the Hearing Impaired program.. The" nature 

of the problem of the student having hearing impairment is much better 

( 

defined than most with the consequence program definition for hearing 
impaired students "might also be more easily specified for other programs. 
Therefore, 'staff members of the hearing impaired programs in the WDPS met 
in after-school sessions to p/epare inputs and later to revive and resub- 
mit additional ones. Program definition^ and indicator selection were the 
# * 

most difficult problems faced by this .group. 

Four programs were defined: (1) language, (2) reading, (3) basic 
living skills, and (4) early intervention. For. each a hypothetical scale 
having four levels was formulateid. "There was no full satisfaction that 
either program definitions or scales were optimally defined for the simu- 
lation experience but it was demonstrated^ that a combination with enough 
credibility to have potential ^use for feedback could be produced. 

Because different staff members in "the department, the director of 
special education and the director of research of the district placed dif- 
ferent values on programs for different levels of student performance not 
one but six sets of input data were initialiy transmitted to Dr.. Cook of 
the DPI for analysis. The inputs, for these analyses and their outputs, 
as well as inputs and outputs for resubmissions, are included in Appendix 
B; the first series are a "test A" series and the second a "test B" series. 
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The first*^ analysis series produced no recommendations for inclusion 
of the basic living skills program as a unit for implementation. Depending 
upon tTie amounts of money available and judgements of the authority who 
prepared input information other programs -might or might not have been 

-implemented at all levels/' Unit entries in the '^proposed implementations" 

c 

coltimns of the output sheets indicate language implementation was usually ^ 
recommended at the three highest levels of preformance. In no case was 
intervention proposed at all levels for any program. These outcomes, 
when they became available, stimulated a great amount of discussion. 
Several conclusions were drawn: ^' ^ 

1, Some program costs were too high, 

2. Not enough money was *'made".available ^ 

3, Perhaps programs should be given different values among themselves, 

4. l^rognoses given for change by students were ^too low; inferentially , 
scaling was not adequate. 

Subsequent decisions ' resulted in ire-estimations of costs for 

programs', the new costs were lower than those originally proposed,. 

New stipulations of values for programs were pi:.epared by some members, 

* * • * ' 

Finally, different amounts of available program monies were also specified, 

new data cards were punched and mailed. 

The resultant B-series of runs did not lead to proposed implementations 
/ ' ' ^ 

for basiQ>. skills. This rather surprising outcome apparently occurred \^ 

/ 1 
because probable outcomes were not estimated to give sufficient individual , 

! 

chances- for improvement. - However, the B-series of runs produced 
recommendations for implementation of each of the other three programs at 
twoyor three levels. Their implementation was recommended in amounts 
of /about $145,000 irrespective of the amounts of funds specified to 



' bd available, which ranged from $150,000 to $250,000. The program proved 

• / ' ^ 

itself to be able to allocate resources and to provide a stimulating 
/' . 39 



organizational structure for the simulation activity* 

Use of Technical Support Systems 

This project was not conducted with the intention of producing a . 
formal test of the adequacy of external support systems. However, it 
did, as a necessary part of its unfolding, involve external support 
elements and found them surprisingly .adequate to project needs. Inquiries 
for information about cost-efficiency models, possible program indicators 
and analysis options were directed to Terry G, Geske who was assigned 
to a coordiriative role by the Wisconsin Research and Development 
Center for Cognitive Learning, Requests for knowledge about special 
education programs and for assistance in data analysis were directed 
to the Wisconsin Department of PuTilic Instruction where they^ were 
promptly responded to; all data analysis for simulation after keypunch 
was efficiently coordinated by that agency. 

Many sources of inforxaation about student acco'unting systems were 
available. The most useful of these was the Minneapolis Public Schools, 
members of which responded, enthusiastically to inquiries. C.^ Thomas 
Randall,, of that agency, came on-site and actively assisted in develop- 
meat of a student registration form. Ttie Institute for Educational , 

Research, Downers Grove, Illinois was a very strong supportive group, 

< 

This service agency for local education agencies in the Chicago area 
possesses unusual capabilities for the initiation of cost^ef f ectiveness 
vork in special education. Its members responded during telephone 
conversations and a site visit. They gave very valuable suggestions. 
It seems clear that external technical support is conveniently 
jivailable for development of cost-efficiency work in special education. 

\ AO. ' , 
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Most if not all needed technical support can be aqulred either within 
Wisconsin or slightly outside its boundaries. If cost-efficiency work 
is to be done on a large scale within the state formal organization to 
provide technical support may be necessary within the university system 
and the DPI. However, in the short range, continued maintenance of the 
level of support available to this project would be fully adequate to 
project needs. 

Initial Study of Resource Allocation 

At the conclusion of the 1974-75 school year each of the 52 staff 
members in special aducation received a questionnaire. Its purpose 
was two-fold:- (1) to bring together each staff member's self-report 
estimate of time spent on each of four general task areas and the percents 
of -time spent on work activities, within each task area, and (2) to 
accumulate individual staff members end-of-year judgements of programs 
that might at a later time be iytiated as cost-efficiency studies 
together with descriptioits of indicators that might be appropriate 
for use with cost-efficiency studies. In late June, when this report 
was prepared, 36 of the 52 staff members in the department had returned 
their questionnaire (69%). Their responses are summarized in tables ■ 
1 and 2; the questionnaire constitutes Appendix C. 

Discussions with staff members during the year had led to the 
definition of four general types of work carried out in special education 
as part of implementation of Chapter 89: (1) screening, (2) prescription, 
(3) servicing, and (4) follow-up. These types of work were defined in- 
terns of operations associated with Chapter 89. The operations seemed to 
be discrete in the minds of the staff members. 
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The staff was' first asked to estimate the time spentj^ each , 
of the four work areas during the 1974-75 school year. Percents x 
allocated by them were to sum to 100 although rounding errors in / 
summarl/zation could' produce a higher 'or lower value.- They were 
then asked to estimate the amount of time they would spend on the 
same category .in 1975-76-af ter which they were asked to give an 
"ideal" time allotment for the same general work types. Their responses 
to this set of items are summarized as average percents and ranges 
of percents in Table 1. This table indicates that , for respondents as 
' a whole, 15, 19, 57, and 10 percents of time were devoted^ to the 
four work types during 1974-75. They estimated slightly less time' 
(3Z) would be devoted to screening and prescription in the next year, 
with mor£ time (4%) devoted to follow-up; it was estimated that the 
proportions of time to be spent in 1975-76 were close to "ideal" times. 

Table 1 

Thirty-Six Staff Self-Report Estimates of Time Spent on 
Four General Types of Work (Percents and Ranges of Percents) 
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42 



46 



Time Allotment 




\ 

- General Type of Work 


'i 


ft 

Screening 


Perscriptiorf 


Servicing 


Follow-Up 


^ Approximate in 
1974-75 


15% 
0-50 


19% 
,2-35 


57% 
35-81 . 


10% 
2-30 


Estimated for 
1975-76 ^ 


• 12% 
- 1-40 


15% 
2-30 


58% 
30-88 


14% 
4-30 


Judgement of 
"Ideal;* 


12% 
OrSO 


16% 
3-40 


20-89 


' 13% 
4-50 



A supplemental analysis of times devoted by staff members indicated 
psychologists and social workers spent more time, proportionately, on 
screening than did other groups of special education^ workers ; those 
working in the mental retardation field spent the lowest proportion of 
time in screening. Members in speech, social work, mental re'tardation 
and hearing impaired areas spetit proportionally more time on prescriptive 
activities while those working with mental retardation and learning 
disability needs spent highest proportions of time on actual servicing 
of needs. Psychologists and social workers spent least proportional 
time on the provision of services. Psychologists spent proportionally 
■more time on fotlow-up as did those working in the area of hearing im- 
pairment . 

- Do the percents that were reported differ from what they might 
have been had all staff members responded tq the questionnaire? An 
answer to this question exists. A supplemental analysis of .the responses 
of persons working in the'same field as non-respondents, was conducted. 
This analysis suggests Table 1 over-estimates total amounts of time devoted 
to screening and under-estimates amounts o.f time devoted to prescri^ption 
and servicing. However, the differences in percents between the total 
group and respondents would be unlikely to exceed 3 -percent for any of the 
-four general work types. Estimates of the amount of time that will 
In the future be spent for the provision of service by nonrespondents 
-suggests the overall figure reported in TaFl'e'l may ^ itself constitute 
an under-estimation of about 3 percent. 

It was previously noted that task-analysis of worker" time is a common 
means of entry for cost-efficiency studies. For task analysis to occur 
it is necessary that discrete work act'ivities be defined by which to 
classify types ^ work that are carried out. The types of work and 

A3 
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activities must have meaning 'for the organization of infbrmation collection . 
and for the initiation of actual programs subsequent to its assembly. 
Seven activities, each of which might have some relevance >o cost- 

. efficiency work, were identified in the course of this, project. These 
seven activities provide role' descriptions for entries in Table 2. They 

- have concern for the professional activities of (1) planning, (2) scheauling 
of work, (3) conduct ,of work, ,'(4) travel, (5) ■ inservi,.ce experiences, 
(6)' slack 'time, and (7) liaison with external ^groups.. Only the "conduct" 
activity has in its meaning provision for coijtact with students; all . • 
other activities are, in a sense, toi purposes of support to student contact. 
Each of the seven activities can be applied/ to. each of the four general 
work types. However, for the staff member who devotes more than a minimum 

, number of hours each week to professional activity it is important to 

■7 . t 

recognize the datar that were collected may not be descriptive of the 

total professional effort of the district staff, ^ • 

Entries in Table 2 are "percents of percents". That is, an entry of 
9Z for planning,' under screening as a work type, indicates 9% of the time 
/ allocation entered in Table 1 (itself 15%) was devoted' to planning for 
screening, — just over 1% of the total work time, 

'Entries in TaT:>le 2 indicate ^staff members estimated they spent from 
61 to 68 percent of their work .time .in the "conduct" category; by work 
group, they spent from 7 to 12 percent of their time at planning, et cetera. 
Estimated ideal times' were very similar to the times actually spent with 
^differences -never, exceeding 3 percent for the combined groups. 

Interpretation of the contents of Tables 1 and 2 must be undertaken 
with great care. While the overall validity of estimates should be quite 
good, the validity of estimates of total time commitments for activities 
having low frequencies of occurrence for a given work type may be very 
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. unreliable, However, the data should have value foe consideration when- 
ever subsequent discussions of sta.ff time allotments occur. Also, staff 
members may find the categories usfeful when classifying their activities 
In weekly planning-reporting sessions.. » 



Design of a Student Accounting Registration -Procedure 

About midway through project development it became evident to the 
authors that cost-efficiency analysis, if it were to be done in the 
WDPS, could not- be efficiently carried out without attention being given 
to a very serious pupil acco.untihg%roblems . More than a dozen forms 
were in/use for the recording of pupil information pertinent tq special . 
education. It was not possible to efficiently relate to eact other;, the • 
different types of information that were collected. More seriou.s^ly, the ^ 
existing .pr'ocedur-es were not at all compatible with information needs over 
the long term-. While- status information about s^tudents at, a specific • 
point in time could be assembled , 'however inconveniently, there was no.-, 
possibility for engaging in longitudinal data collection without the 

commitment of excessive hours of personnel time. Furthermore, there 
were' gaps in the types of information that WoJxd be needed axid some 
types of information, while being collected, were not being organized In 
the most useful manner. Therefore, close attention was directed to this 
overall problem"with the result that a new, -comprehensive student 
registration, procedure was developed. 

The procedure that was adopted could not have been developed without 
, external assistance. There has been, as was noted earlier, a great amount 
of experience accumulated by laxQei^ educational agencies which developed 
Student accounting systems. Therefore, it became a goal of the project 
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to acquire from others' the best of their experience and to use that" ^ 
experience to design a suitable local system. The search for this 
experience led to review of procedures used by the Phildelphia j-^ Minneapolis, 
Milwaukee and St. Louis public schools, ftr. Gary Holloway, of the DPI,- 
provided necessary informa'tion about the student accounting procedures 
maintained within his agency. 

All forms in use for reporting purposes by. the m?S and current-data- - ' 
^collec-t-ibn-proc-e-dures were reviewed as part of this activity. At its 
culmination a new form was drafted and shared with members of the Special 
Education Department. After review by them it was submitted to general 
administration personnel in the district and a conclusion was drawn' that ' 
It "was in a form "suitable for general adoption. 

The form that was adopted is a modification of one currently in 
usXfor student registration in the Minneapblis Public Schools. It calls 
for u^e of a student identification number both to make ;easier the accessing 
of information apd to "provide for improved anonymity where anonymity is 
lmportant\ Students names, addresses^ and family status are recorded on 
It with entries o'f this type being capable .of providing some of the demo- 
graphic information for describing 'system inputs. Spe'cial coding for _ . 
special educatio^^^ro grams in the district and- the State of Wisconsin were • 
prepared. Additionk modifications of the Minneapolis documentation 
enable "coding for special education at locations other than the traditio.nal 
classroom, for recording' of the tuition status of students in special education 
who are not residents of the district and for notation of the full range 
of transportation codings .for vtherregular student body and for the 
handicapped. This form, the contents of which appear as Appendix p, will 
be printed on a form having a harder stock paper background separated by 



carbon from a lighter front sheet. Vhe first sheet will, upon registration 

■A'7 



of a student in a building, be transmitted to a central location for ^ 
coding and transmission to keypunch after which a fraction of the data 

V 

will be stored. The harder backup sheet will be retained at -the building 
level for use in it^srecord system Cos t-oftTiese. forms, when printed 
afv^I^Iiiiri^^^Iiir^y "f or the district has been estimated by vendors at. • 
about '2.5 cents each. ' * , - 

It is anticipated at the time of this writing that this procedure , 
will be approved for adoption by the Board of Education in the nextf 
school year. If this is the case a central file will be established 
during the falll and winter of the 1975-76 school year after which update 
and -reporting systems will aiso be applied. Adoption of this system 
will not itself reduce costs of activities of special education infor- 
mation production but should enable much more to be accomplished with 
the current expenditure level. 
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Summary and Reconunendatf&Tns 



A model for cost-efficiency analysis of, special education has be^en ^ 

v.. ■ , " " iS^ 

produced during conduct 'of this development project.. If contains inter-. ^ 
related practical and theoretical elements. The fbllowine practical 
*\ problems came to-be'viewed a3 the more '•sdrious'-obstacles to fututfe cost- 
efficiency work in special education: * 

1, Lack of reliable, valid indicators by which to assess outcomes. 

2, Limitations "in the number of programs that can be ^designed . 
with sufficient specificity to permit their imp.lementatipn 
for close cost-efficiency analysis, 

3, Lack of existing cost allocation practices which can 
efficiently produce cost information about programs* 

' ' . 4, Lack of eJ:ficient pupil accout^ting systems-; ^ \ 

5. Lack of personnel 'experience with cost-efficiency procedures. 

^Howeyer, in the steps taken to' go beyond theory and .into practice, 

progress was made toward resolution of several problems : ^ - " - ' 
• • ^ 

,1. A necessary basic student accounting, system was designed 
for the district, J - ^^-r 



2, A simulation procedure for resource allocation was pilot 
tested and found to have .potential value. for inservic^ . 
conduct of efficiency analysis, 

i 

3% Task analysis classifications were developed for special 
" ' * education, . " ^ ' - - ' 

4. The task-^'analysis classif ic^ation enables some cost- 
effici'tincy study, to be done without adoption of 

cumVer^ome microanalysis of staff timCrJ'^ ... • * 

Some conduct of cost-^ef f iciency development, work can' continue in 

the district Without external assistance, -However, external knowledge , 

and other resource support will be needed if. progress is to be rapid. 

The most important resource needs that continue to exist are the following 

^ Resources to make efficient the recording of student 
program information. ' 
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2. Resources ^or validity studies of screening and prescriptive 
practices* ^ 

3* Support for external .data analysis of input and output 
, information. * 

Discontinuance of the ESEA Title III funding prociedure in effect 

the time, this project was conducted makes uncertain the best course to 

follow for assembly .of additional develppment support. However, because 

needs and potential outcomes have been quite '^clearly identified it seems 

• f 

certain^ progress will not now be terminated even though additional 

• » , 

■ »,* 
external support could be productively applied. 

, • ' \ . 
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APPENDIX A 



THE STUDENT-CHANGE MODEL 
OF AN EDUCATIONAL SYSTEM 
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/ 

A DecSsion i\/fodej- 

' Practical. Applications , 

/ 



Oy John J. Cook, Ph.D. 
Coordinator, Rc$<flrch ^ 

Division for Mnndicippcd Children, DPI ^ ^ 

Managers and administrators in special education at both 
the state and local level are. called upon to make various 
kinds of decisions. The most typical one concerns the 
allocation of limited resources to programs developed to 
attain certain educational goals for the community. Ip its 
simplest sense the decision process involves two parameters, 
the utility or value of an outcome or goal, and the 
probability of attaining this outcome. These combined 
factors then determine the course of action or,' in a 
practical sense, the amount of money which should be 
allocated to the various programs. 

This article outlines a procedure which* allows the admin- 
.Istrator, in conjunction with his advisory committee or 
other decision^making body, to make the necessary sub- 
decisions on the basis of which the optimum allpcation of 
resources can be specified. The decision bits are fed into a 
computer program which has been modified by the author 
for use on the Univac 1108 at the U.VV. Computing Center, 
but based on the original work of Drs. Harry Harmon and 
Robert Patrick' of the Educational Jesting Service. The 
program in turn specifies the optimum allocation of the 
money. In this article no attempt is made to explicate in 
detail the underlying theory and mathematical formula- 
tions. Rather, should the reader be interested in more 
information about the model it is suggested he contact the 
author. Should sufficient interest be generated then a one 
or two day workshop could be considered. ^ 

An Example 

In the following example an ottempt is made to be as 
realistic as possible. However, data from problems already 
run 2re being used as a mattei of convenience, so some 
distortion of reality might be evident. 

Let us assume that the individual in charge of EMR 
programs in the state has uuen receiving S1'],250,G58 per 
year to fund programs for the EMR. Let us assume further 
that the administrator is given an additional S1,0C0.000 to 
beef up state-wide programs for the EMR. Exactly what 
form this "beefing up" process will take is unknown at this 
time. It could be curriculum additions, different types of 
adjunctive services, in-service training »ind so forth. The 
problem for the administrator is how best to distribute this 
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money among his various proyrains. The term "program" 
lends Itself to many definitions but for our purposes, the 
6MR programs will adhere to the chronological definitions 
of Early Education, Primary, Intermedi.ite, Junior and 
Senior High School. 



Basic Data • 

The basic data associated v/ith e^ch of the programs at this 
time are as follows: / 





Maximum 








Enrollment 


Program 


Cost 


Program 


Potential 


Cost 


Per Pupil 


Early Education 


1048 


$ 3,735,500 


$3,564 


Primary 


1800 


$ 4,t93,987 


$2,330 


Intermediate 


1310 


$ 2,359,224 


Si. 801 


Junior High 


2291 


$ 1,065,350 


$ 465 


Senior High 


7977 


$ 2,896,597 


$ 363 


TOTALS 


14,^26 


$14,250,658 


$ 988 



Once again the rc.':d8r is warned that the data are fictitious 
so some discrepancies with reality can be noted; thing: such 
as the high funding of preschool education which is just 
starting to move or the disproportionate number of pupils 
in Senior High School. 



Current Status of Programs 

To make judgments, choices or preferences among several 
levels of performance in the different programs, an assess- 
ment of current status must be made. In the Primary and 
Intermediate progiams performance could be indexed I'y an 
achievement test such as the Peabody Individual Ac!iieve* 
ment Test {PI AT), If the goals are other than academic 
achievement, alternate indices have to be devised. Such 
would be the case in the Early Education, Junior High and 
Senior High programs. The scale for each of these progr;ims 
could be an overall index or composite of several indicators. 
Of course, indicators would vary depending on the pro- 
gram. For instance, in Early Education the indicatofs might 
be adequacy of physical plant rate Low (L). Medium (M) or 
High (H). adequacy of r^edical attention, rated L. M, or H, 
and qualifications of educational personnel, rated L, M, or 
H. The overall scale might then be 1 if no Hs. 2 if 1 H,3 if at 

• 1 



least 2 Hs, Junior and Senior High School on the other 
hand would .have different indicators which for our 
purposes could be academic achievement, behaviors related 
to work study and general -deportment. Thus the complete 
$calc could be handled as with Early Education, The maUW 
generated from this information would be as follows with 
the proportions based on numbers in each program at each 
level on the scale. 



Scale 





Low 






High 




Program 


r 


2 


3 


4 


Total 


Etrly Education 


.40 


.35 


.25 




1.00 


Primary 


.23 


.26 


,27 


.24 


1.00 


Intermediate 


•18 


.28 


.28 


.26 


1.00 


Junior High . 


.50 


.20 


,30 




1.00 


Senior High 


.60 


. .20 


:20. 




1.00 



All the weights must add bp to 1.00. It is to be noted that 
the PrtrTia,ty.and Intermediate programs also have four levels 
on the scale.' ''Since academic achievement is the major 
concern the scale levels for .these programs, could cor- 
respond lo: at or above age equivalent on Total Test score 
of FIAT (4), up to one year below (3), one to two years 
below (2) and more than two years below (1 ). 

priorities Across Programs 

At this point, the administrator would call together his 
advisory committee to ascertain the relative value of each 
of the programs, i.e., what are the priorities? The programs 
are then rank ordered and weights are assigned in terms of 
the committee's concern for the attainment of the pro- 
grams' objectives. Once again all weights must add to 1.00 
and in addition, as required by the model, the toi) priority 
program is given a weight equal to the combined weights of 
all the less valued programs. This situation could be as 
follows: 





Weighted 


Progran) 


Concern 


Primary 


,50 


Early Education - 


.25 


Junior High 


.10 


Senior High 


•10 


Intermediate 


.05 


TOTALS^ 


1.00 



In a group setting such as a committe^e there arc several 
ways to obtain the weights. They can be independently 
assigned and an average"obtained. Discussion followed by 
consensus could also be used. A more time consuming and 
perhaps more valid approach is the Delphi technique for 
establishing consensus. Once the decision making model has 
been made known to the committee, the series of question- 
naires usbd to successfully approximate true consensus 
could be used prior to the actual meeting (For moio 
information on Delphi- technique, see^'Nation's Schools, 
July, 1973, p. 29-32). 

Priorities WiUlin Programs 

The next' job for the committee is to indicate theii 
preference or concern for moving pupils out of each level 
(below the top onel^^as against moving them out of the oth- 
er levels. As before, all weights must add to 1.00 and the 
top scale level for each program is given a weight equal to 
the weights at the lower scale levels. This latter constraint is 
most adequately resolved in many situations by giving the 
top scale level a weight of .50, thereby making available the 
widest possible range of weights .for the lowerlevels. 

The within program priority weights \^o\j\6 look like this: 



Scale Levels 



program 


' 7 


2 


3 


4 


Total 


Early Education 


.35 


.15 


.50 




1.00 


Primary 


.25 


.15 


.10 


.50 


1.00 


Intermediate 


.25 


.15 


.10 


.50 


1.00 


Junior High 


.35 


.15 


.50 




1.00 


Senior High 


.25 


.25 


.50 




1.00 



Practically speaking what' the committee is required to do 
can be illustrated by the Early Education Program. Their 
weights are saying in effect that it is more than twice as 
important to them to moVe the pupils in programs, rated L 
or M in terms of physical plant, medical attention or 
personnel qualifications to\the next level (2) where one 
indicator at least is rated H, \hao to move the pupils from 
programs with one H rating to those with two or n!oie H 
ratings. 

Estimates of Probable Outcome 

The final chore for the committee before the computer 
takes over is to estimate the probable outcomes after one 
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year or for some other specified time span. These probable the interventfon. The matrices resulting would appearaike: 
outcomes are estimated without the jntervention and with this: 



probable Outcomes 



Without Intervention 



Early Education 





From/To 


1 


2 


_3 






i 


fin 


.15 


.05' 






9 




75 


.20 






o 




.05 


.95 






Primary 












From/To 




2 


• 3 


4 




• 1 


.90 


.07 


.02 


• -^.01 




2 


.05 


.80 


.10. 


.05 




3 


.01 


.04 


.80^ 


.15 




4 


.00 


•.05 


.10 


.85 




Intermediate 












From/To 


1 


2 


3 


*4 




1 


.90 


.08 


,02 


.00. 




2 


.02., 


.93 


.05 


-V. .00 




3 


.01 


.04 


.93 


^ .02 




4 ^ 


.00 


.01 


.04 


.95 



Junior High 



From/To 


1 


2 


3 


1 


.80 . 


.20 


.00 


2 


.05 


.03 


.97 


3 


.00 


..03 


.97 


Senior High 








From/To 


1 


2 


3 


1 


1.00 


.00 


.00 


2 


.05 


.90 


.05 


3 


.00 


.03 


.97 



With Intervention 



From/To 


^ 1 


2 


3 








.40 


.20 




z. 


no 


.50 


.50 




o 


nn 


.00 


1.00 




From/To 


i 


2 


^ 3. 


4. 


1 


.50 




1 

. 1 3 


0^ 


Z 




.60 


30 


.05 


*> 

o 




06 


.5$ 


.40 


4 


.00 


.03 


.17 


* .80 


From/To 


- 1 


_L 


'3 


4 


1 
1 


50 


.25 


.15 


.10 


2 


.00 


.65 


.25 


.10 


3 


.00 


.00 


.70 


.30 


4 


,00 


.00 


.00 


i.od 


From/To 


1 




3 




1 


.60 


.30 


, .10 




2 


.00 


.60- 


.40 




3 


%00 


.00 


^ 1.00 




From/To 


1 


. _2_ 


3 




1 


.75 


.20 


.05 




. 2 


.00 


.65 . 


.35 




^3 


. .00 


.00 


/.OO 





An interpretation of tliese matrices seems warranted, tn the 
Primary ptogram. for instance, without the Intervention 
90% of the pupils are expected to remain at level 1 (more 
■than two years b.elow their age equivalent on the PIAT), 
with the program only 50% are expected to remain so; 80% 
are expected to remain at level 2 (one to two^years below 
age equivalent) without intervention and G0% with inter- 
vention; 80% are expected to remain at level 3 (up to one 



year belov/ age equivalent) without and 52% with inter-, 
vention; 85% are expected to remain at level 4 (achieving at 
or above age equivalent) v.ithout and 30% v/ith interven- 
tion. Similar interpretations can be applied to the rest of 
the matrices, 7% at'e expected to go from level 1 to h:vel 2 
without and 30% with intervention. 3% from 1 to 3 withou*t 
and 15% with, 1% go to 4 without and 5% with the 
intervention, and so forth. 
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Computer Output 

All the ddla specified in the above sections are entered into 
the computer in the specified format. Output would then 
consist of the ' ollowing: 
Available Funds: $1 ,000.000 



Concern Max. Proposed 
^Program . Across Enrollment Enrollment Cost 



Primary 


.50 


1800 


900 


5209,970 


Early Education 


.25 


1048 


1048 


S373.550 


Senior High 


.10 


7977 


3191 


$115.86^ 


Junior High 


.10 


2291 


2:591- 


$106,535 


Intermediate 


.05 


1310 


969 


$174,583 
$980,502 



In addition to the above, the computer also provides the 
levels within the program toward which the Input indicates 
the intervention should be directed: 



Program 


1 


•2 


Primary 


> 

1 


0 


^ Early Education 


1 


1 


,Seriiro Higlv 


0 




JunioV High ^ 


i 


. 1 


Intermediate 


1 • 


1 



Proposed Implemeatations 



Thus^ all levels of tlie Early Education and Juiior High 
programs aie of equal priority whereas levels 1 and 3 of 
Primary, 2 and 3 of Senior High and levels 1, 2 and 3 of the 
Intermediate program are of top priority. 

Final Comments ^ 

In order to use this approach to resource allocation, it is 
apparent that the people involved must .be thoroughly 
acquainted with their.school system and with the nature pf 
the inlerypntion^f the estimates called for nrp to have any 
basis in' reality. Also, raigo scale field testing of llic 
procedure has not been undeitiikcn so arguments for its use 
A are a little difficult to develop. 

» 

The procedure is meant to be quita general in Its'appl'ca- 
tion being limited only 'By the ingenuity of the user. VVhila 
the example used illustrated its*us'j on a statewide basis. It 
Is equally amenable to use by the small L.E.A. For instance. 
Xitle I funds can be used in a variety of situations which 
'could bevdcfined as prdgrams as the. term is used in the 
example. Using the decision model would foico i clarifica- 
tion in thinking about the project as welj as determining the 
optimun\3llocation of the funds. 

It would seem that v>^ith the <advent of block grants, a 
procedure such as the deci:>ion model might wyl! become 
indispensible in administrative decision making. In any 
event, should theunsponse to the ai tide be adequate, it ccn 
be tried in some, practical situations put forth by, the 
participants.- / 
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SPECIAL EDUCATION COST -EFFECTIVENESS SURVEY 



TIME ALLOCATION 

In discussions with many o.f you we have triSd to learn how we mi^^^t 
''classify use of personnel time in special education. Since much cost- 
effectiveness work is built up from task analysis , this is an important 
oarea for us to consider. * 

Your reactions to our initial presentations suggest we can talk 
about four general types of work: 

Screening - involved the identification of children who might need 
special education services. This work phase continues until a 
decision is made (yes or no) that a child has a special need 
requiring service, i^e. they are to receive Chapter .89 or 
other services. 

Prescription - is the series of tasks by which the body of knowledge 
of your field of specialisation is applied to plan remediation, 
or treatment. Careful considerations lead to specification of 
a program to others if they are to carry it out. - 

Servicing - is the process of remediation. Students - receive the ^ 
treatment prescribed for them. This might occur in onertd-one 
or grouping arrangements. 

/ ' if ' 

Follow-up - occurs as treatment is concluded. It may involve post- 
testing, assessment conferences with others and parental visits. 
A summary report is prepared, .the contents of which suggest 
whether additional services will be. needed. 

How do you estimate percents- of your time have been distributed across 
these types of work this year? How do you estimate they will be distribute 
in an ideal, established program including Chapter 89 services for this 
.district? In the spaces that follow please enter your estim^es^qf percent 
for each of the past, next and ideal years.* 



. '1. 1 » . * ' General* type of •WorU 





^Screening 


Prescription 


Servicing 


Follow-up 


fotal 


My^appro^^^^^te^^ 
1974-75 time 
allotment^ 


.V ' — 


4 \ 


■ % 


7. ■ 


100% 


My estimated 
1975-76 .tim^ ^ 7 
allotments^^a^' ^ 




% 




% 


100% 


An "ideal 
allotment ^ 
^of time 


' r 


♦ 


% 


% 


100% 


^ . . / 
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Each of the foOr general types of special education work by itself 
has tasks or activities involved in its conduct. There sefem to- be seven 
such activities: ' * * 

Planning - involves selecting a strategySy which to conduct screening, 
prescription, servicing or follow-up. Answers are formulated 
for such questions as: How "shall we go about this? Wiat , 
guidelines should we follow? What considerations should we 
keep in mind? Much administrative time may fall into this category. 

Scheduling -involyes arrangements to get work done, time setting, 
notifying people, setting up agendas.- \. 

^ Traveling - is getting places where work -is to be done. 

;onducting - is the action phase of screening, et cetera. M- teams 
' Wet /children -are ''worked with, parents are counseled. 

H^^vinR In-service - can occur for each general work area although it 
^ tends not to be scheduled in exactly that way. In-service 

tends to involve* more^.that one work phase which means it may 
have to*be broken out for estimation purposes. 

Slad k time - is time not used productiVely . ^This is a" normal 

feature of employment which arises because schedules are broken, 
\or expected and planned-for work has not arrived, 
* \ ' ' • 

External liaison - is a commoji feature .in professionai work. Meetings 
^ .vlfith staff, of other agencies, speaking engagemenjjis or providing 
fbr visitors are examples of this ac,tivity. 



I. 2' 

With th^se definitions in mind, how do you estimate your screening 
time (if you \did screening) has been spent this, past year? In an'^ 
ideal operation for this district, how do you estimate screening time , - 
might be spent? Please enter percents to answer these questions in the 
columns that follow, 

. * Activities Associated with SCpIeENING 



ERLC 



\ .-^ 

Activity 


Estimated Percents of 
Time involved during 
1974-75 \ 


Estimated Ideal •[ 
-Percents -^of Time 


Planning for ^ 


% 




% 


Scheduling foV 


% 


% 


Conduct of » 




% 


traveling for 




% ' 


Having inservice 
about 


''\ 




■ / ^- 


Slack Time 


% 




Exterti<kl Lia-ison 




% 


Tatal 
1 \ 


' 100 % 


100 % 



How do you estimate your prescriptive tirife (if you did this) was 
spent this past year? In an ideal operation for 'this district, how do 
you estimate prescription time might be spent? Please use the columns 
that follow to answer these questions. 

Activities As'sociated with PRESCRIPTION , • 



. A^ctivity 


* Estimated Percents of 
Time involved during 
197A-75 


Estiniated Ideal 
Percents of Time 


Planning for 




X' ' 


Scheduling for 


X 


X 


Conduct of 


1 




Traveling' for 


% 


- . X 


Having inservice 
about » 


i 2 


X 


Slack ^Time 


X 


X 


■ .t 

External Liaison - 


X 


%■ 


total 


■ 100 X 


. 100 X 



I. A. . ... 

How do you estimate your service time was spent thi*s past year? 



Activities Associated with SERVICING 



Activity 


Estimated Percents of ' 
Time involved during 

197A-75 . ' 


Estimated Ideal 
Percents of Time 

0 


Planning for 


'■X 


X 


Scheduling for 




X 


Conduct of 


X 


" ' % 


Traveling for 


X 


% 


Having inservice 
about 




, . X 


Slack Time 


X 


X < 


External Liaison 


% 


% 


Total 


• 100 X 


100 X 


<• 

• • 


^1 C.3 
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r 
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I. 5 . - . . , 

How do you estimate your follow-up time (if you did this) was 
spent\ this past year? In an ideal operation for this district, how. do 
you estimate follow-up time might be spenf? Please use the columns 
that follow to answer these questions*. 

Activities Associated with rOLLOW-IiP- ' 



♦ 


1 

i 
/ 

ACuivxty 


'Estimated Pcrcencs oC 
Tt n\p "involved diiriiif' 
1974-75 


fcisciir^ced Ideal 
Porcents of Tir,;c 




.Planning for 


Z 


i 

% 




Scheduling for 


Z 


z 




•Conduct of 


V 
At 


•/ 




Traveling for ^ 


% 


z 




Having inservice 
. about 


% 


z 




Slack Tine 


z 


? z 




External Liaison 


z 


Z" 




Total' 


100 v. 


100- % 




I- 6 


•w1 


■ 1 < 

"i 



Do you believe the activity classifications we are trying to iTse 
do a good job? If not, what changes might you suggest?^ 
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INDiaiORS 

. Whit -(three at most) indicators do,, you believe may be reliable 
and valid enough to be used for outcome /assessment in your field ot 
specialization? (If it x^ill help, see the attached description of 
what features a useful "indicator" shoujld have). 



Indicator' 1: 


'i 
















7<S 










V ) 




u 


..Indicator -2; 








/ 









■ / 



Indicator 3: 




III, PROGRAMS 




What two distinctive special education .programs tha^t i^nvolve yoAir 
^ield might you advocate for cost-effectiveness cons£<lerjation? In 
responditig, please provide enough information \o permit a reader to 
u'nderstand what its' goals might be and- trqwit might funcjtion\ (See 
thi attached description of "program" to m^aerstan\f eatpres a program\ 
sho^d have if its cost-effectiveness is to be vigq^rously stud^ied). 




Please use the back of this page to further elaborate your ideas if more 
space is needed. 

i 8 1 C.6 
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APPENDIX A.l 

K 

INDICATORS; EDITED DEFINITIONS AND SELECTION CRITERIA 

i • ^ 

"Indicator" is not a new term that jneeds to be added to , the vocabulary 
of most educators. It has been used for a number, o'f years in the fields , 
of business, economics, and social research. I^at may be riew tb educators^^ 
is the use of the term in association with assessment. Therefore, there - 
is a need to clarify its definition and identify acceptable criteria for 
its definition and identify .acceptable criteria for its selection and. U|,6.. 
Oregon's project Has made an effort to. do both' of these things. 

■po^ur purpose an "indicator" can be defined as: ^ 

A descriptor in quantifiable .terms, of the status of a signiEicaht 
conation or variable which provides evidence useful for an analysis 
of progress toward, a goal or objective. 

Three i'mpart-ant elements appear in -the definition: 

(1) The expression is quantifiable - data does exist, or can be 
, .be collected, to ;how "how much" of the indicator exists. 

(2) ' The condition or variable that is described has, by general 

,agreement, a relationship to the goal with which it is associated. 

(3) The measurement is associated with a point in time. 

To illustrate, an indicator which contains these three elements is:. f 

The nmboA of high 6diooZ 6zyUou ^ho did volantzQA mnk In a 

' / ; (1) ' 
cormunltLf 6oclal a gmaij duULcng thz ^cixool ijdOJry s^' 

^ (2) ; ^ ^ 

To be of greatest use in assessment, indicators should be": 

(1) Derived from reliable and valid data. 

(2) Derived from data that will continue to be collected so 
that comparisons over time may be made. 

(3) Derived from data for which the measurement techniques have 
stability over tijne. 

It should be recognized that indicators do not .describe the desirability 
or quality of the progress reported. Such judgments rest ultimately with 
the population as a whole. 

A performance indicator describes a measurable or observable behavior 
or variable used to' determine program af f ectiveness pr efficiency. Data 
may concern: (a) student performance scores, or (b) a program variable 
such aa instructional process or program variable such as instructional 
procesi or availability of learning experiences Examples are: 

Student test results 
Observable behavior 

Number of students attaining performance requirements 
of a course 

Number of learning situations outside of school that *are 
available to students. 



- APPENDIX A. 2 
PROGRAMS; ELEMENTS OF A DEFINITION 
USEFUL FOR COST-EFFECTIVENESS 



We have talked with most of you about the meaning for the term program 
as it must be used to fmprove our ability to do cost-effectiveness work. 
By "program" in this sense of usefulness 'we must have in mind. not a broad 
area program, such as EMR or SLO^ but an operation within an area by itself 
or across two or more such broad areas. 

Personnel time, which is' our most valuable resource, must be easily 
separated our into intervals ■ dev6ted to a usefully defined program versus • 
time spent on other programs. Similarly, materials and other resources must ■ 
be easily cla^ssif iable by use within, a single program rather than in many. 

-What a program is should be ,so clear smd definite that there is little 
■ or no ambiguity about it. Whether personnel eime and other resources are 
being expended for a program or not should be easy to explain. In summary, 
it must be easy to know what' services , time and money are supporting. - 

For example: 

(1) "^A pre-school progi;am for children witl> hearing impairments is 
distinctly different from a general program for all children 
with the handicap. ^ 

"'^ (2) 'An effort to provide for improvement of self image would 

probably not be distictive enough to permit clear-cut cost 
- allocation. This would certainly be true if the effort was 
thought to be part of all that transpires in- a broad "area 
program. ^ 
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